Figure 1: Due to Compatibility issue with GitHub Pages, please go to https://samuellee19.shinyapps.io/SFQ_Final_Project/ to use interactive dashboard.
Project Overview
Background:
The Stop, Question and Frisk program is a practice, utilized by New York City Police Department (NYPD), of temporarily halting, questioning, and, in certain cases, searching civilians on the street for weapons and other dangers. It is also called “Terry Stop,” named after the Supreme Court case Terry v. Ohio (1968).
While it had been used previously by the police as a method to deter crime, it was championed by Mayor Bloomberg when he was elected in late 2002. Since then, there has been a lot of controversy surrounding the Stop and Frisk practice, it has been cited as being discriminatory, and many have claimed that there are racist motivations in choosing who gets stopped and then frisked.
The use of Stop, Question and Frisk practice is often endorsed with the Broken Windows Theory, suggesting that even low-level crimes and civil disorder leads to more serious crimes in urban environments. In fact, after NYPD officer Adrian Schoolcraft made extensive recordings on the department’s Stop and Frisk policy, numerous civil rights organizations, such as NYCLU, raised a concern that the program unfairly targets certain minorities, such as African-Americans and Hispanic-Americans. Similarly, the Guardian reporter also noted that “since 2002 (the Kelly/Bloomberg years), over 35% of officers feel high pressure to write stop-and-frisk reports…felt less pressure to obey constitutional constraints.”(Eterno and Silverman 2013). Due to this, in 2008, a state judge ruled that the police department had to publicly release information regarding this practice.
A major turning point was the 2013 court case Floyd v. City of New York and a subsequent NYPD mandate that requires officers to thoroughly justify the reason for making a stop. After Bloomberg’s term ended, Bill de Blasio took over the office and pledged for a major reform on NYPD policies.
Project Objectives:
While we have some information on New York and the different areas, we’d like to gain some intuition on how the stop and frisks are distributed throughout New York. Therefore, we begin by creating a visualization tool to visualize the stop and frisks by year and region in New York. Next, many critics of the Stop and Frisk policy claim that it isn’t actually effective in deterring crime. Therefore, we’ll be visualizing how this program has changed the rate of crime. Finally, we seek to determine how just this practice is, and whether any policy changes have impacted how equitable it is. Therefore, we begin by determining if areas with higher crime rates also have individuals being stopped and frisked more, as one would expect. We then will see if any changes in policy and administration have affected how fair this policy is. While much analysis has been done on the discriminatory practices of actually stopping the individuals, not much has been done on how individuals are then chosen to be searched or frisked. We therefore analyze what are the main factors into determining this over three “eras” of policy change, which we go into more detail about in the relevant section, to then compare how the policy changes have affected this practice.
Methodologies
Data Used
The data is found in the following Drive folder: http://bit.ly/DSFinalProjectData (Spatial Data on all periods can be found in https://www.dropbox.com/s/sz41qn8ez1kipbs/SQF0418_SP.zip?dl=1)
There were 2 pieces of data we used, the first was data about Crime in New York by Precinct from 2000-2018. The data came from the New York State Police Department (https://www1.nyc.gov/assets/nypd/). The second is the Stop and Frisk data collected by the New York Police Department from 2003-2018 for each individual they stopped. For 2018, this data came from New York’s ACLU branch (https://www.nyclu.org) but for the remaining years the data came from the New York State Police Department (https://www1.nyc.gov/assets/nypd/). The Stop and Frisk data for 2017-2018 was in a different format than the data from 2003-2016, Therefore, we’ll be using data from 2003-2018 for visualization and part of our analysis. However, for our analysis related to the factors in determining whether or not an individual is searched or frisked, the data we use is from 2003-2016 to allow for comparisons between the years.
Visualizations
For this project, we mostly used library(mapdeck) to create screengrid and hexagon objects to show density of Stop and Frisk events across our timeline, as you can see from the above dashboard.
Figure 2: Spatial Points of Stop and Frisk Data (2017-2018) with Plotly and Mapbox
This is a basic interactive scatterplot of 2017-2018 Period data created on Plotly and Mapbox. We can see that spatial points are dispersed across all of New York City boroughs. While this scatterplot allows us to see where Stop, Question, and Frisk practices are happening, there is no way to be certain which areas have higher rate or density of occurance. In order to resolve the issue, we created a screengrid interactive that you can zoom in and out, tilt, and move around:
| Control Options | Description |
|---|---|
| Moving | Arrow Keys |
| Zoom | Scroll Wheels or Trackpad Zoom |
| Tilt/Orientation | Control (Command on Mac) + Shift + Arrow Keys |
It is also important to mention that the density is expressed by color. From the map, we can see that there are significantly more density in northern parts of and above Manhattan, such as East Harlem, Fordham, and Bronx. Since East Harlem and Fordham have significant minority population, it would be interesting to see whether race has an influence over decision-making of NYPD’s Stop, Question, and Frisk practice.
You can also conduct further interaction with more data points and options with a dashboard that we created on the top. You can also access the dashboard at https://samuellee19.shinyapps.io/SFQ_Final_Project/. (NOTE: some features could take some time to render due to size of our data)
Effect of Stop and Frisk on Crime Rates
To begin, we wanted to analyze whether stop and frisk actually seemed to be having any impact on the crime rate seen in an area. Therefore, using the data described above, we decided to visualize the effect of the stop and frisks on the crime rate of an area, by precinct. To do this, we first normalized the number of stops per precinct over a year. We then found the rate of change in the amount of crime per precinct for each year we had data on. Finally, we graphed this information, as shown below.
Figure 3: Change in Crime Rate vs Amount of Stop and Frisks per Precinct
Figure 4: Change in Crime Rate vs Amount of Stop and Frisks per Precinct by Year
As we can see, there isn’t always a clear pattern in a relationship between the change of the number of crime and the scaled amount of stop and frisks seen in a precinct. If the stop and frisks did have an effect on crime, we would have expected there to be a strongly negative correlation. However, there doesn’t seem to be any relationship between these two values. When we separate the values by year, we can also see sometimes there were some negative trends, which means the crime decreased as there were more stop and frisks, but also have some years with a positive trend meaning that areas with more stop and frisk also had an increase in crime. Therefore, there doesn’t seem to be a clear connection between the number of stop and frisks and the change in crime of an area. Therefore, this suggests that stop and frisks didn’t seem to serve the purpose they set out accomplish – the data doesn’t suggest clear alleviations in the crime rate.
Factors for Stop and Frisk Individuals Being Searched or Frisked
Because the stop and frisks didn’t seem to be helping the crime rate in an area, we next wanted to figure out exactly how these stop and frisks were being conducted – what factors lead to someone being more likely to be chosen for a stop and frisk?
Relationship Between Crime and Stop And Frisk
The first factor we analyzed which may lead to there being more stop and frisks in an area is how much crime happens there. We’d expect areas to have more crimes overall to also have a higher rate of stop and frisks, because the purpose of the stop and frisks would be to try and target these high crime areas. We therefore worked on visualizing a relationship between the amount of crime in an area and the amount of stop and frisks in an area, by year. We did this by standardizing the amount of crime per precinct by year, and then standardizing the amount of stop and frisks conducted per precinct by year. We then plotted these values, as shown below.
Figure 5: the Amount of Crime vs the Amount of Stop and Frisks per Precinct
Figure 6: the Amount of Crime vs the Amount of Stop and Frisks in per Precinct by Year
Through these graphs, we can see again there is no trend in values nor clear positive trend; in fact, there is a slightly negative trend. This means that even though areas had a high amount of crime, there wasn’t a corresponding high amount of stop and frisks happening in that area. One may think this is due to the stop and frisks decreasing the amount of crime in that year, but as we saw in the above analysis there was no relationship between these two variables. Therefore, we can see that stop and frisks don’t affect the crime rate, and it’s also not based on where there is more crime. Therefore, there must be other factors in play, in determining which individuals were more likely to be stopped and frisked.
What Factors Actually Matter: Analysis of Effects of Change on Policy
Because crime rate didn’t have a clear relationship with the amount of stop and frisks, we then turned to the data of the stop and frisks to see what were the main factors which lead to individual being frisked or searched. The way we did this was to use a random forest model to find given the variables in the data whether or not someone would be frisked/searched or not. We then found the most important variables for each of the random forests. We made a random forest for every other year, and then found the important factors on them. We then averaged the importance across the years in the different groups of years. We chose the years 2003-2007, 2008-2012, and then 2013-2016 as our three eras for which to find which variables were important. We chose these years because of policy and administration changes. In 2007, there was a ruling which said that the stop and frisk had to be publicly available. Following this ruling, there was a lot of analysis on the data and public outcry over discriminatory practices. In 2013, Bloomberg’s mayoral administration came to an end. Bloomberg was a major proponent for stop and frisk, and following de Blasio’s entrance as mayor, there were large policy changes in how the stop and frisk was conducted. One policy change, due to a court case, enforced that thorough justification must be given for why someone is stopped and searched.
The 6 most important factors for whether or not someone was searched or frisked after being stopped are:
| Time Period | Six Most Important Factors |
|---|---|
| 2003-2007 | Furtive Movements, Inappropriate Attire for Season, Suspicious Bulge, Refusal to Comply with Officer’s Directions, Actions Engaing in a Violent Crime, and Other |
| 2008-2012 | Furtive Movements, Actions Engaging in a Violent Crime, Inappropriate Attire for Season, Refusal to Comply with Officer’s Directions, Suspicious Bulge, and Other |
| 2013-2016 | Furtive Movements, Actions Engaging in a Violent Crime, Risk from Action of Engaging in Violent Crime, Suspicion of Weapons, Refusal to Comply with Officer’s Directions, Inappropriate Attire for Season |
For the first 2 eras, the top 6 reasons were the same, but the order in which the reasons occurred was different. In the third era, Actional of Violent Crime and Suspicion of Weapons swapped with Suspicious Bulge and Other. Some of the interesting parts of these values are that half of the reasons in first 2 eras are ambiguous or subjective. Having inappropriate attire could be caused by many things, having a suspicious bulge is a very subjective reasons, and within the other category, there were generally very subjective descriptions. In the third era, these variables seem much less subjective and much clearer indicators of a risk of crime. The inappropriate attire variable is still there, but it’s now become the least important of the 6. The other 2 subjective variables have been swapped with variables that are much less subjective.
Therefore, while there doesn’t seem to be any clear discriminatory practices in how individuals were chosen to be searched or frisked, there did seem to be a lot of subjective factors that went to it, especially when Bloomberg was mayor. It seems like the policy enacted in 2013 did have an effect in ensuring that the justification for why someone is stopped, searched, and frisked was thorough.
Conclusion
To begin, stop and frisk doesn’t have a clear effect on decreasing the amount of crime an area sees. Further, there isn’t a connection between the amount of stop and frisk and the amount of crime which does happen in an area. While past research has focused a lot on discriminatory practices in the way individuals are chosen to be stopped, our research, which focusses on the method individuals are chosen to be searched or frisked, found that were not any glaringly discriminatory practices. However, during the Bloomberg administration, a lot of the factors which determined whether or not an individual was searched or frisked seemed extremely subjective. During de Blasio’s term, the factors are more objective and substantiated by evidence.
Eterno, John, and Eli Silverman. 2013. “Mike Bloomberg’s Fact-Free Defence of Stop-and-Frisk.” The Guardian, September. https://www.theguardian.com/commentisfree/2013/sep/11/stop-and-frisk-michael-bloomberg.